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ABSTRACT 

Research paradigms are not the proper channel for 
educational evaluation. Evaluation and research differ in nany areas, 
including purpose, methods, goals, groups, and desired outcomes. 
Research is strictlj controlled, has the purpose of gathering 
information and making generalizations about completed studies or 
events. Evaluation is a process asking for feedback from groups as 
they exist, not under controlled conditions. Evaluation seeks 
specifics that show what is happening in an event trhile research 
explains causes. (SN) 
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Traditional research paradigms are not adequate for doing educational 
evaluation. This view is held by Stake. Cuba. Stuftlebeam and Scriven among 
others and stems primarily from the fact that both the assumptions and goals 
of traditional research, perhaps better termed "conclusion-oriented research'' 
(Cronbach and Suppes) are different from those of educational evaluation, which 
might be termed "decision-oriented research," (Stuff lebeam or Cronbach), and 
thus a paradigm produced on the basis of the assumptions and goals of the 
former are of necessity and by definition inappropriate in assessing the goals 
of the latter. 

Let's examine briefly some of the assumptions and goals of conclusion- 
oriented research. First, research has as Its primary goal the advancement 
of knowledge or "Truth." It strives to advance and extend knowledge (Cuba). 
Furthermore, data collected from a research paradigm must be internally valid 
(Stuff lebeam) in order that it be as generalizable as possible (Stake). To 
achieve all of this a researcher employs the principles of randomization of 
subjects and treatments, control of extraneous or interacting variables and 
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However, this is fundamentally different from what educational evaluation 
strives. Cuba says the evaluator is trying to devise and test some practical 
solution to an operating problem. He is concerned with resolving a number of 
problems simultaneously if he can. He is concerned also and perhaps most 
importantly with the need to be able to refine and/or adjust his solutions 
continuously. Unlike data produced by an experimental design, data which is 
usually £0^ hoc (Gubaj Stuff lebeam) evaluation data needs to be continual in 
order that, as Cronbach points out (and Cuba, Stufflebeam and Hastings would 
all appear to agree) ongoing decisions regarding an educational program may be 
made while the program is in progress and not after it has been terminated. 
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In fact I according to Stufflebeanii '*«#«thc application of experimental design 
to evaluation problems conflicts with the principle that evaluation should 
facilitate the continual improvement of a program." (Stuff lcbeam» p. 49), 

Furthermore, evaluation deals with the "worth of oomething," (Stake) f^r 
the "valuing of something," (Scriven) or with "...the use of human judgment," 
('"lass) and not just the description of something. In the conclusion-oriented 
aradigms, however, this point is precisely to be avoided at all coyts. 

Let's also sxamine more carefully the techniques of research and why they 

are inadequate for evaluation. Regarding the notion of generalization, there 

is a basic difference. In fact even the title of Stake's article articulates 

this difference: "the need for limits." In evaluation. Stake argues, the purpose 

of inquiry is for "specification" whereas the inquiry in research is for 

"generalization." He is saying that the purpose of and results of evaluation 

in fact should not be generalized atid cannot be geueralizticl. There is a "need 

for limits" regarding the generalization of evaluative data. Evaluators are not 

concerned that findings hold over different schools, over different communities 

and over replications (Stake). Obviously this is not true of findings In con- 

t 

elusion-oriented research since in order to "extend knowledge" generalizations 
have to be made, the wider the generallzabllity, the better. 

To achieve control over the threats to validity such as history, matura- 
tion, reactive arrangements and so on, the researcher tries to use randomization 
to assign students to treatment and control groups. He tries to hold all other 
variables except treatment variables equal during the duration of the experiment. 
The treatments cannot be modified during the course .if the experiment, Again, 
this is exactly what evaluators do not want and in fact do not and cannot have. 
Seldom if ever can evaluators exert the kind of cont;*;o3. which is demanded by 
research. (That he doesn't want to is another point.) The evaluator Is usually 
working with a specified problem in a specified setting with specified subjects. 
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Me cannot as a rule randomly assign subjects or treatments, rjn control groups 
control for the various throats to validity mentioned in Campbell and Stanley 
and so on. In addition he does not want to be representative of others, but 
rather wants to look at the given program for its own value as it is perceived 
by the decision makers of that program (Gubaj Stuff lebea...) . 

Assuming thv. such tight control can be exerted, as both Cuba and Stuffle- 
beam point out. ,„d extraneous variables are held in check, then the findings 
which result wHj „ot even be generalizable to the school or program at hand 
for in a school or program in the real educational world, these so called 
extraneous variables operate freely. It is important therefore to know how 
programs operate under real world conditions and not under the carefully 
controlled conditions of a laboratory situation (Guba). Stake concurs on 
this point: 

....as soon as we exercise a reasonable degree of exoerimental cnnhrni 
asp^cu "nrtan'r^^"' ""i^bility in the prograrand h td h ' 
oroer™ tlT^l r ' i= «lt"ed. Many an educator find the 

(Sta'ke™ WeS! p^^'^f ^"ed £o longer the program he wanted to know Tout. 

There are yet other differences which exclude the utility of experimental 
designs. Gagne writes that most learning experiments for example have been 
concerned with the effectiveness of single units of a curriculum, or at the 
most a very few units. A paradigm such as pre-post test, no control design or 
a Solomon 4 Block or whatever is fine for examining a single unit, it obviously 
fails when looking at a larger, ongoing constantly changing program with inter- 
acting variables over which there is no control. Stake concludes his argument 
this way.* 

I™?a'li'l° J^J''"' " t™^™ental choices to be scientific, 

To fin"r:ut^r(^J:^er»l5f'p"2r "eU^leed...- 

The former represents conclusion-oriented research, and the latter, evaluation. 
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In auromary then I would like to quote from Egon Cuba: 

...an evaluation paradigm that emphasized control when invited Inter- 
ference Is needed; that prevents attention to more than one problem 
at a time;,., that provider only terminal data; and that renders 
impossible the crucial requirement for continuous adjustment and 
refinement, simply cannot be judged very useful by the practitioner. 
Indeed, he must find such a paradigm not only useless but in fact 
crippling to his purposes. (Cuba, 1969, p, 4) 
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